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ABSTRACT 



This report, intended for local and state policy makers and 
practitioners, summarizes the findings and implications of class -size 
reduction (CSR) in California. The report's findings provide a comprehensive 
picture of California's CSR initiative during its first 2 years, covering 
such topics as implementation, resources, teacher qualifications, teaching 
practices, parent involvement, and student achievement. The paper offers a 
synopsis of the legislative background of CSR and describes some of the 
responses to implementation of the law. For most schools, the greatest 
obstacle to implementation was a lack of space, followed by insufficient 
funding, insufficient time to plan, and a lack of qualified staff. The report 
offers an overview of how the teacher workforce changed with CSR. Although 
teaching practices did not change significantly with implementation, teachers 
reported being able to spend more time with problem readers and less time 
disciplining students. Parents had more contact with teachers and expressed 
more satisfaction with their children's education. Small gains in children's 
academic achievement, regardless of ethnicity, income status, or 
English-language ability, were evident. However, CSR led to overcrowding in 
many schools, and poorer districts were unable to attract as highly qualified 
teachers as the better-off schools. (RJM) 
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YERYIEW 

In the summer of 1 996, the California 
Legislature and then-Governor P6te 
Wilson adopted a sweeping reform to 
revive academic performance by lowering 
class size in the early grades from a maximum 
of 33 to no more than 20 students per teacher. 



With virtually no planning time, school 
districts managed to put hundreds of thou- 
sands of students in small classes by the time 
school started, just six weeks after the legisla^ . 
tion (SB 1 777) passed. By 1 999-00, the fourth 
year of the program, virtually all I ” and 1'^ 
graders will be part of the program, plus an 
estimated 95% of kindergartners and 90% of 
3*^ graders. 



THE CSR RESEARCH CONSORTIUM 






Spurred by the belief that state policy makers would be best served 
by an evaluation that is comprehensive and multi-year; five organiza- 
tions combined forces in 1 996 to form the California Class Size 
Reduction Research Consortium. The Consortium is led by the 
American Institutes for Research (AIR) and RAND. Other partners 
in this first evaluation include EdSource, Policy Analysis for California 
Education (PACE), and WestEd. The State Board of Education 
accepted the Consortium’s proposal for a four-year evaluation, with 
financial support from the state budget, several private foundations, 
and the U.S. Department of Education, Office of Educational 
Research and Improvement (OERI).The Consortium released its 
first set of findings on June 23, 1 999. 




O 



California’s class size reduction (CSR) ini- 
tiative is thought to be the largest state educa- 
tion reform in history, now costing over $ 1 .5 
billion per year. Given that price tag, the stakes 
and expectations for the program are high. 

What has been found so for? Class Size 
Reduction in California: Early Evaluation 
Findings, 1 996-1998, the first report of a 
four-year, legislatively-mandated evaluation 
shows evidence of successes. But it also 
points to unintended consequences that are 
cause for concern. 

On the positive side, the evaluation of 

CS[^ Dm its second year ( II 997-98) finds: 

■ Relative to students in larger classes, 3’’^ 
grade students in smaller classes showed, 
on average, a small positive achievement 



gain.The level of gain was similar for all 
groups of students, regardless of ethnicity, 
income status, or English language ability. 

■ Districts rose to the challer^e of implement- 
ing CSR quickly, finding 23,500 additional 
teachers in two years. The result: over 1 .6 
million students in kinder^rten to 3"^ grade 
were placed in reduced size classes. 

■ Teachers in smaller K-3 classes reported 
spending more time working individually 
with problem readers and attending to the 
personal needs of students.They also spent 
less time on discipline than teachers in 
larger classes. 

■ Parents of students in reduced size classes 
had more contact with teachers and were 
more satisfied with their children’s education. 

Other findings from this two-year 

assessment of CSR are less positive: 

■ To implement CSR quickly, some districts 
with already overcrowded focilities had to 
take classroom space away from other edu- 
cational programs. 

■ Because space was particularly tight in dis- 
tricts with high proportions of low-income, 
minority, or English language learner (ELL) 
students, these districts were slower to 
implement smaller classes in all four grades, 
K-3. Districts that could not reduce class 
size quickly received proportionately less 
CSR revenue than districts that could. 

■ Many districts also found that the cost of 
creating smaller classes exceeded their 
CSR revenues. This was particularly the 
case for districts with the most low- 
income, minority, or ELL students.To make 
up the deficit, they diverted resources 
from other programs. 

■ Most important, schools with the highest 
percentage of low-income, minority, or ELL 
students were unable to attract as highly 
qualified teachers as other schools. 
Statewide, the K-3 teacher workforce 
increased by 38%, but the overall qualifica- 
tions of K-3 teachers declined. 

Such findings imply a need to make mid- 
course policy adjustments to help increase the 

program’s benefits. Key among the implications 
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is the need to bolster teaching — ^from 
recruitment to preparation, support, and 
professional development — and to contin- 
ue to find ways to support school con- 
struction. Equity-related issues are also 
raised, such as creating incentives for good 
teachers to work in schools that most 
need their expertise and changing the fund- 
ing formula in order to give districts more 
flexibility in the use of CSR funds. 

The report begins with some back- 
ground on California’s class size reduction 
law. The second section describes imple- 
mentation and the program’s impact so 
far on California’s enormously diverse 
schools, student populations, and educa- 
tional programs. Next is a description of 
changes to the teacher workforce that 
occurred with CSR, along with compar- 
isons of teaching practices in smaller and 
larger classes. Parent involvement and sat- 
isfaction are then summarized. The next- 
to-last section provides details of the 
student achievement findings.This report 
concludes by outlining policy implications. 

Getting started 

How It Happened 

In late spring of 1996, California’s public 
education system faced numerous chal- 
lenges — classrooms overcrowded from a 
1 2% increase in K- 1 2 enrollment over the 
previous five years, a growing shortage of 
qualified teachers, and indications that stu- 
dents who needed to learn English or who 
came from low-income families were 
falling steadily behind. In fact, most Cali- 
fornia 4* grade students were achieving 
poorly, since their reading scores on the 
National Assessment of Educational Progress 
(NAEP) put the state tied for last place 
among 39 participating states. 

As the state’s Legislature entered its annual 
budget debates that same year. It encountered 
an unusual situation. After a number of very 
lean years, the state’s economy was booming 
again. Because of the provisions of voter- 
approved Proposition 98, this revenue windfall 
had to be spent primarily on public schools. 





COMPARING CALIFORNIA TO TENNESSEE | 


Although programs to reduce class sizes have been tried in several states, the best 
known was Tennessee's “STAR” project The early success of that experiment was 
a primary impetus behind California’s law — even though conditions in schools in 
this state, especially the huge and diverse K-3 enrollment make direct compar- 
isons inappropriate. Here are essential differences between the two states: 


Tennessee 


California 


■ a controlled experiment in a 
limited number of schools, with 
teachers and students randomly 
assigned 


■ Statewide implementation in 
four grades (K-3), beginning 
with 1“ and 2""^ grades 


■ class size reduced from 22-26 
to 13-17 


■ class size reduced from an aver- 
age of 28.8 (maximum 33) to a 
maximum of 20 


■ adequate space for smaller 
classes 


n extremely limited space due to 
rapidly growing enrollment 


■ credentialed teachers 


■ existing shortage of teachers, 
some hired without credentials 


■ nearly all students spoke English 


■ nearly one-third of students 
were English language learners 


■ standardized objectives for 
English and mathematics 


■ standards and objectives under 
development 


■ existing state tests aligned with 
standards 

1 


a no statewide test until 1998; 
it was not aligned with new 
standards 



Bolstered by polls showing strong public 
support for smaller classes and encouraged 
by the governor, legislators seized on the 
findings of an educational experiment in 
Tennessee called Project STAR.These findings 
indicated that reducing class sizes produced 
significant improvement in student achieve- 
ment.The results were particularly promising 
for low-income and minority students, v/ho 
benefited almost twice as much as other stu- 
dents. For all these reasons, reducing class 
sizes in California’s burgeoning primary 
grades seemed a logical way to invest the 
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SOURCES OF DATA FOR 



THE CSR EVALUATION 




Information for the Consortium’s first evaiuation report came from three sources: 

■ data submitted annually to the California Department of Education (CDE) by school districts 
and teachers. 

■ written surveys completed by superintendents, principals, teachers, and parents. 

■ classroom observations by the Consortium team. 

Information from the CDE for 1996-97 and 1997-98: 

■ CBEDS,the Comprehensive Basic Educational Data System, collected by the Educational 
Demographics unit each October from county offices of education, school districts, and schools. 

■ Language Census (R-30), a March count in each public school of students who need to learn 
English (ELL) and those who are now fluent in English (FEP). 

■ femily income information, collected in October; about students who are eligible for free or 
reduced price meals or whose parents are in CalWORKS (formerly Aid to Families with 
Dependent Children, or AFDC). 

■ California Special Education Management Information System (CASEMIS), a twice-yearly 
“snapshot" of special education for students with disabilities. 

■ current expense of education, a Business Services Division calculation of expenditures per 
pupil (excluding capital outlay for facilities) in each district 

■ J-7 Operations and Facilities report about CSR implementation, from the School Facilities Division. 

■ STAR (Standardized Testing and Reporting) scores from spring 1 998. 

The Consortium’s first round of surveys was mailed in April and May 1 998. The sample 
included: 

■ 1 25 school districts, 99 of which agreed to participate; 87 superintendents returned the sur- 
vey, for an overall response rate of 70%. 

■ 625 schools from the 1 25 districts. Of the 432 that agreed to participate, 336 principals 
responded, for an overall rate of 54%. 

■ 1 ,485 teachers from the participating schools.The teacher response rate was 66%. 

■ 2,1 12 3'^ grade parents, two each from the classes of the surveyed teachers. Slightly more 
than half responded. 



■ impiementation and 
iresource allocation. 
How quickly was CSR 
implemented among differ- 
ent districts? How did the 
funding mechanism for 
CSR influence which dis- 
tricts received the money? 
How did the implementa- 
tion of CSR affect other 
programs? 

■ teacher qualifications. 
How has CSR affected the 
overall qualifications of 
California’s teachers? Are 
qualified teachers distrib- 
uted evenly among schools 
serving different student 
populations? 

■ classroom practices. 

Has CSR changed the way 
teachers teach language 
arts and mathematics? Are 
they covering more con- 
tent? Are they teaching in 
different ways? 

■ parental involvement. 
How has CSR affected 
parental involvement in or 
attitudes about their chil- 
dren’s education? 

■ achievement. Are stu- 
dents in smaller classes 
achieving at higher levels 
than those in larger class- 
es? Do students from dis- 
advantaged backgrounds 
benefit more from CSR? 



additional revenue in a reform that could 
improve public education. 

Evaluation Aims to 
be Comprehensive 

In examining CSR, the Consortium pursued a 
comprehensive approach, investigating the 
program’s relationship to students, teachers, 
schools, and districts. The main research ques- 
tions address: 



Findings from the first 
two years of implementation ( 1 996-97 and 
1 997-98) begin to answer these questions. 
However, the results outlined in this summary 
report are preliminary; over time the evalua- 
tion will elaborate upon the answers to these 
questions. The Consortium will not complete 
a full summative report for another three 
years. Furthermore, the analysis is necessarily 
limited to issues that can be addressed by 
using reliable data (see box above. Sources of 
Data for the CSR Evaluation). 



O 
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Figure I 



PERCENTAGE OF STUDENTS IN REDUCED SIZE CLASSROOMS 
BY GRADE LEVEL AND YEAR 




Grade Level 



1996- 97 

1997- 98 



Participation increased in the second year of the CSR program, with over 1.6 million K-3 students in reduced 
size classes. 

Source: California Department of Education. Retrieved February 24, 1999 from the World Wide Web: 

httpy/www.cde.ca.gov/ftpbranch/sfpdiv/dassize/facts.htm 6/99 



CSR IMPLEMENTATION 
RATES AND RESOURCES 
VARY 

In 1 996-97, the Legislature and the governor 
chose a powerful incentive for schools to 
reduce class size in the primary grades: a 
grant of $650 for each child in a class of 20 or 
fewer. Schools had to reduce grade first, 
then 2"^ followed by 3^^ grade and/or kinder- 
garten. The state also offered $25,000 for new 
or renovated space for classes.The following 
year the per pupil grant was increased to 
$800, and the facilities grant to $40,000. 



Dastrkts Moved Swiftly to Create 

Smaller Classes... 



The Strong financial incentives — coupled with 
the perceived benefits of small classes — creat- 
ed an attractive opportunity for most school 
districts. Many willingly created smaller class- 
es, even if the cost exceeded their allotted 
'jnds or took space away from other 
£ J^(]“ims. As Figure I indicates, by CSR's sec- 



ond year, districts had risen to the challenge, 
placing over 1.6 million students or nearly all 
I and 2"*^ grade students and nearly 70% of 
all California’s kindergarten and 3^ grade stu- 
dents in smaller classes. 



...But Some Districts Had 
More Difficulty 

The evaluators found that different types of 
districts phased in the program at different 
rates. Rural and suburban districts were bet- 
ter able to reduce class sizes immediately. As 
a consequence they received more CSR rev- 
enues on average per eligible K-3 student for 
both facilities and operations than urban dis- 
tricts (see Figure 2). Further, districts with 
high percentages of minority students were 
less likely to have adopted the program in 
CSR’s first and second years, so they received 
less CSR revenue on average per K-3 student 
compared to districts with low percentages of 
minority students (see Figure 3). 

Districts with higher percentages of 
Hispanic and ELL students took longer to 



Certain school 
districts had to 
delay reducing 
class sizes in all 
eligible primary 
grades largely due 
to lack of space 
and insufficient 
CSR funding. 
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Figure 2 



DISTRICT CSR FUNDING IN FIRST 
TWO YEARS OF IMPLEMENTATION 
BY URBANICITY 



Figure 3 



DISTRICT CSR FUNDING IN FIRST 
TWO YEARS OF IMPLEMENTATION 
BY PERCENTAGE OF 
MINORITY STUDENTS 




Urban icity 

■ CSR Facilities 96-97 
CD CSR Operations 96-97 
C3 CSR Operations 97-98 



$800 
8 $700 

g c $600 
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S ■§ $500 

$400 
a $300 
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Percentage of Minority 
Students in District 

■1996-97 CSR Facilities 
EZH 1996-97 CSR Operations 
01997-98 CSR Operations 




The chart displays the amount of CSR funding received per “eligible student” (all students in grades K-3). A higher dollar amount indicates 
greater participation in CSR. If all K-3 students were in reduced size classes, the amount of operations money would have been $650 and 
$800 per eligible student in 1996-97 and 1997-98, respectively. 

Source: California Department of Education. Retrieved February 24, 1999 from the World Wide Web: httpy/www.cde.ca.gov/ftpbranch/sfpdiv/classize/facts.htm 6/99 




phase in CSR. This slower rate of implementa- 
tion also occurred, but not as noticeably, in 
districts with high proportions of low- 
income' students. 

Faster implementation, on the other hand, 
happened in school districts that started out 
with more money to spend. Under Califor- 
nia’s long-standing school finance system, 
some districts receive more total revenue per 
pupil than others. Because these districts 
could reduce class sizes more quickly, they 
received more CSR funding sooner. This left 
the districts that had less money to spend 
from the start even further behind in their 
efforts to establish smaller classes. 

As a result of the differences in district 
implementation rates, many at-risk students 
who stood to benefit the most from CSR, 
according to the Tennessee experiment, were 
less likely than their peers to have been 
placed early in a smaller class. 

^Students are referred to as low-income in this report if state 
records classify them as receiving public assistance in the form of 
Aid to Families with Dependent Children (AFDC) or its successor 
in California, called CalWORKS. 




Lack of Space a Major Obstacle 

Beyond analyzing financial data, the evaluators 
surveyed district superintendents and school 
principals about their experiences with the 
class size reduction program.These local 
administrators paint a mixed picture of the 
beginning years of CSR. 

Topping their list of Implementation prob- 
lems were facilities and space constraints. Due 
to enrollment increases many elementary 
schools were already overcrowded when the 
class size reduction law was passed. Over 80% 
of the principals who had not yet implement- 
ed CSR in all four grades cited a lack of space 
as the problem (see Figure 4). 

The challenges were greater for urban 
districts, where some schools were land- 
locked; they literally had no space in which to 
add a classroom. 

In many schools, principals usurped space 
formerly used for other purposes. In the 
beginning, the space most commonly bor- 
rowed to accommodate smaller classes was 
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from special education, and this continued 
into the second yean By 1 997-98 many of the 
new classrooms had formerly housed child 
care, music, arts, computer labs, libraries and, 
to a much lesser extent, teacher preparation 
rooms and gyms. Many schools retrofitted 
existing rooms and/or purchased portable 
classrooms. At one point the overwhelming 
demand for portable classrooms temporarily 
exhausted the supply. 

Districts Confronted 
Budget Trade-Offs 

The second most frequently cited reason for 
not phasing in all four grades was insufficient 
money — ^the gap between CSR funding and 
anticipated district costs for CSR implementa- 
tion. Districts more likely to report such a 
deficit were those with large enrollments, 
those classified as suburban, or with high ELL 
or high minority student populations. 

In the first year, over half the superinten- 
dents said they had to spend money from 
their operating budget to cover the full costs 
of operating smaller classes. Even with the 



second-year increase in the per pupil grant, 
over 43% of districts still reported a shortfall 
of funds.Two-thirds of the districts that were 
large or had higher proportions of minority 
or ELL students reported a deficit during 
CSR’s first year; over half reported a deficit 
the second year. 

Asked where they found the money to 
make up the gap in funding, superintendents 
reported that the supplemental money most 
often came from reducing facility maintenance 



Figure 4 



REASONS PRINCIPALS GAVE FOR NOT COMPLETING CSR 
IMPLEMENTATION IN ALL ELIGIBLE GRADES BY END OF 1997-98 



Implementing Other Reforms 



Could Not Recruit Sufficient Staff 



Insufficient Time to Plan 



Insufficient Funding 



Insufficient Space 




Percentage of Principals 



Principals reported lack of space as a substantial obstacle to reducing class size. 
Source: CSR Consortium 1998 Survey of Principals 



6/99 





OPINIONS OF PRINCIPALS AND SUPER- 
INTENDENTS ABOUT THE EFFECTS OF CSR 



Boosted primary grade teachers' 
enthusiasm for other reform 

Brought in teachers with new ideas 
that enhanced reform efforts 

Created new professional development 
needs that had to be addressed first 

Used facilities that were 
needed for other reforms 

Diverted my attention from 
most other reform efforts 

Caused us to delay 
or postpone reform activities 

Took everyone’s time away 
from other reform efforts 





88.3% 



Superintendents 

Principals 



J 



20 40 60 80 100 

Percentage of Superintendents and Principals 



The chart displays the percentage of principals and superintendents who “Agree” or “Strongly Agree” with various 
statements about the effects of CSR on other education reform efforts under way in their schools and districts. 

Sources: CSR Consortium 1 998 Survey of Superintendents and Principals 6/99 




and district administration cost. Superinten- 
dents said they tried to protect educational 
programs, although nearly 40% did report 
important cuts. Some cut libraries, computer 
labs, or music and arts. A smaller number of 
districts, according to the surveys, reduced 
professional development funds, special educa- 
tion, or child care, but funds were rarely cut 
for bilingual, after-school, early literacy, and 
sports programs. 

In stark contrast, some districts realized a 
funding surplus from CSR support. According 
to superintendent surveys, these surpluses 
occurred most frequently in small districts, 
both rural and urban ones, and those with 
less than 30% ELL students. Less likely to 
report a surplus were suburban districts, large 
ones, and those with smaller concentrations 
of students from low-income families.The 
surpluses, according to the superintendents, 
subsidized many of the same things that were 
being cut in less fortunate districts — mainte- 
nance, libraries, computer labs, early literacy, 
and staff development programs. 



Schools also found it challenging to main- 
tain adequate learning environments. More 
than half the principals surveyed said that CSR 
made it more difficult to provide sufficient 
instructional supplies and furniture and to 
keep buildings clean and repaired. 

Lack of Teachers Also 
Posed Challenges 

Only about 23% of principals said that not 
being able to recruit sufficient staff hindered 
their implementation of smaller classes in all 
eligible grades.When they did phase in class 
size reduction, more than half of the principals 
said that hiring credentialed teachers was dif- 
ficult. Hardest to find were substitutes and 
teachers with bilingual or special education 
credentials.The larger the district or the high- 
er the concentrations of minority, low- 
income, or ELL students, the greater the 
challenges in hiring teachers. Twice as many 
principals in these types of districts reported 
a recruitment strain. 
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Even So, Administrators 
See Pluses in CSR 

Most of the responding administrators said 
their schools or districts were involved in 
other educational reforms. When asked about 
the interaction of CSR with these reforms, 
some superintendents, and far fewer princi- 
pals, reported a negative effect (see Figure 5). 
Over half the superintendents and principals 
declared CSR a plus for school reforms 
because of teachers* enthusiasm and an influx 
of teachers with “new ideas.” 

Superintendents and principals surveyed 
said that, despite considerable challenges, the 
administrative time they spent on CSR was 
not a problem for them.They did, though, 
acknowledge that CSR diverted their atten- 
tion, preempted space, and created new 
demands for teacher professional develop- 
ment activities. 



ELL students were more likely to make cuts 
in other educational programs and were less 
able to keep their schools clean and well 
maintained. And, as the next section discuss- 
es, many of these same districts had schools 
that struggled to hire teachers with appropri- 
ate credentials.These resource and teacher 
quality differences among districts could 
extend for many years into the future. 

Some immediate policy implications relat- 
ing to these issues are outlined in the back of 
this report.The multi-year evaluation will also 
continue to monitor these issues over time. 

Profile and 

DISTRIBUTION OF 
TEACHER WORKFORCE 
CHANGED 



The percentage 
ofK-3 teachers 
newest to the 
profession and/or 
lacking full cre- 
dentials grew sub- 



Funding and Facilities Issues 
Need Further Attention 

The “flat** funding method chosen by the 
Legislature for the class size reduction pro- 
gram gave all school districts the same 
amount of money for each student in a 
reduced size classroom. The alternative, a 
sliding scale based on district need or some 
other criteria, was too complex to negotiate, 
given the tight legislative time frame. 

Now, three years later, it is evident that 
the need for facilities, along with the flat fund- 
ing mechanism, hampered some school dis- 
tricts from entering the program right away 
or from implementing it fully in all four 
grades.The result was unequal implemen- 
tation rates among districts and differ- 
ences in CSR revenues. 

As more districts phase in CSR, these 
initial gaps will presumably continue to nar- 
row. Yet the findings show that students 
who were minority, low-income, or ELL 
have not yet benefited fully from CSR, pri- 
marily because the funding formula did not 
take into consideration individual district 
cost and space constraints. 

Urban districts or those with high con- 
centrations of minority, low-income, or 



Despite considerable obstacles, districts set 
about recruiting teachers in a short time. In 
just two years, they increased the teacher 
workforce in grades K-3 by 38% or 23,500 
teachers. A noteworthy result of this hiring 
increase is that K-3 teachers now are slightly 
more likely to be male and Hispanic — ^two 
traditionally underrepresented groups in the 
teaching profession. 

However, the surge in teachers hired was 
accompanied by a disturbing overall decline in 
teacher qualifications.Teachers in K-3 class- 
rooms in 1997-98 tended to be less experi- 
enced, less educated, and less likely to be fully 
credentialed than the teachers in these grades 
prior to CSR. 



stantially with the 
implementation 
ofCSR, 



HOW TEACHER 



WERE ANALYZED 



CSR's influence on the number; characteristics, and distribution of teachers was 
another focus of this evaluation.To determine the impact on the teacher work- 
force and where teachers were placed, the Consortium looked at 1995 to 
1997 data collected from the Professional Assignment Information Forms 
(PAIF) of the California Basic Education Data System (CBEDS). Teachers’ pro- 
fessional experience, level of education, and credential status were compared in 
different categories of elementary schools, e.g., schools with different percent- 
ages of ELL, minority, and low-income students. The data did not permit com- 
parisons to be made between CSR and non-CSR classrooms. 
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Figure 6 
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QUALIFICATIONS OF CALIFORNIA’S K-3 TEACHERS 
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1995 

1996 

1997 



Percent not 
Fully Credentialed 



Percent with 
Bachelor's Degree 
or less 

Teacher Qualifications 



Percent Novice 



Characteristics of the K-3 teacher 
workforce changed with the addi- 
tion of many new teachers for the 
reduced size classes. (Novices 
are teachers with three years or 
less teaching experience.) 



Note: 1996-97 percentage of novices 
omitted due to an unusually high propor- 
tion of missing experience data. 

Source; California Basic Educational 
Data System, Professional Assignment 
Information Forms 6/99 



Figure 1 



K-3 TEACHER CREDENTIALS IN SCHOOLS WITH HIGH TO 
LOW PROPORTIONS OF LOW-INCOME STUDENTS 





Year 



The percentage of K-3 teachers who do not have a credential increased for all groups of schools, 
especially for those with the highest proportions of low-income students. 

Note: Differences between the top and bottom quartiles or groupings In the same year are statistically significant at the .01 
level. Differences between years in the same quartile or grouping are also statistically significant at the .01 level. 






Source: California Basic Education Data System. Professional Assignment Information Forms 
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Teachers’ 

Qualifications Have 
SDopped Statewide 

As Figure 6 shows, before 
CSR about 1 7% of K-3 
teachers statewide were 
“novices,” that is, they had 
been teaching for three 
years or lessTwo years 
later that figure jumped 
to 28%. Likewise, the pro- 
portion of teachers who 
had the least education, a 
bachelor’s or no degree, 
increased from about 1 7% 
to nearly 23% statewide. 
The most notable shift in 
teacher qualifications was 
the percentage of teachers 
without full credentials. 
Over the same time period, 
this figure increased from 
under I % to over 1 2% 
statewide. Many of these 
teachers were hired with 
emergency permits, waivers, 
or internship credentials. 
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Figure 8 



K-3 TEACHER EDUCATION IN SCHOOLS WITH HIGH TO LOW 
PROPORTIONS OF LOW-INCOME STUDENTS 




Year 



The percentage of teach- 
ers with minimum educa- 
tion, i.e., no more than a 
bachelor’s degree, is high- 
est in schools with higher 
proportions of low-income 
students. 



Note; Differences between the 
top and bottom quartiles or 
groupings in the same year are 
statistically significant at the .01 
level. Differences between years 
in the same quartile or grouping 
are also statistically significant at 
the .01 level. 

Source; California Basic 
Educational Data System, 
Professional Assignment 
Information Forms 6/99 



The evaluators do not yet know what 
impact this reduction in teacher qualifications 
may have on student performance but will 
continue to examine the issue to the degree 
that the data allow comparisons to be made. 
While increased experience, education, and 
credentials are not a guarantee of “quality” or 
effective instruction, a growing body of 
research evidence indicates that such charac- 
teristics are important. 

Certain Schools Lost Ground 
in Attracting the Most 
Qualified Teachers 

While the teacher shortages forced many 
schools to hire uncredentialed teachers, the 
most qualified new teachers could pick and 
choose among schools.These teachers were 
less likely to be found in schools with higher 
proportions of low-income or ELL students, 
both before and after CSR. 

Teacher Credentials. Figure 7 shows the per- 
centage of uncredentialed teachers among 
schools with varying degrees of low-income 
students before and after CSR. Quartile I 
includes schools with the smallest percentage 
of low-income students, while at the other 
^ highest percentage. 

ERIC f I 



Prior to CSR, schools with the highest 
percentage of low-income students had only 
about 1% more uncredentialed teachers than 
schools with the lowest percentage of low- 
income students. By 1 997-98, that gap had 
widened to just over 15%. In other words, 
elementary schools with the greatest propor- 
tion of students from low-income families 
ended up with a greater proportion of teach- 
ers with incomplete credentials. 

Schools with large enrollments, classified 
as urban, or with a greater proportion of 
minority or ELL students also appear to have 
been at an increased disadvantage in hiring 
credentialed teachers. Conversely, attracting 
qualified teacher candidates appeared to be 
easier in smaller schools, schools classified as 
rural or suburban, and schools with a smaller 
percentage of minority or ELL students. 

Teacher Education. A similar story unfolds 
regarding teacher education levels among dif- 
ferent types of schools. In 1 995-96, teachers 
in schools with the highest percentage of low- 
income students were less well educated than 
teachers in schools with a smaller percentage 
of low-income students. As Figure 8 shows, 
by 1 997-98 that gap had grown to 18% 
between schools in Quartiles I and 4, com- 
pared to 15% in 1995-96. 
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TEACHER EXPERIENCE, IN SCHOOLS 


1 


Figure 9 
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Year 

The percentage of novice teachers, those with no more than three years of teaching experience, 
increased for all schools. 

Note: 1996-97 percentage of novices was omitted due to an unusually high proportion of missing experience data. 
Differences between the top and bottom quartiles or groupings In the same years are statistically significant at the .01 
level. Differences between years in the same quaiHie or grouping are also statistically significant at the .01 level. 

Source; California Basic Education Data System, Professional Assignment Information Forms 6/99 



centage of K-3 teacher experi- 
ence levels narrowed between 
high and low-poverty schools 
(see Figure 9). It also narrowed 
or stayed the same between 
other categories of schools 
analyzed. 

Bilingual Certification. Finally; 
Consortium researchers ana- 
lyzed the distribution of teach- 
ers who hold one of California’s 
two credentials for teaching 
ELL students — the Cross- 
Cultural Language and 
Academic Development 
(CLAD) and the Bilingual 
Cross-Cultural Language and 
Academic Development 
Credential (BCLAD). Although 
the numbers of CLAD and 
BCLAD-certified teachers 
were increasing statewide, after 
CSR schools with the highest 
percentages of ELL students- — 
arguably those schools with the 
highest demand for teachers 
with bilingual credentials — 
attracted a smaller proportion 
of these teachers. 




That pattern persists when comparing 
teachers’ education levels among schools 
with other characteristics. Although schools 
in all categories hired more teachers with 
only a bachelor’s degree, the schools that 
were larger, classified as urban, or had the 
greatest concentration of ELL students 
attracted a lower proportion of teachers 
with higher education levels. 

Teacher Experience. Meanwhile, the propor- 
tion of novice teachers — those with three 
years or less teaching experience — also grew 
among all categories of schools. In the year 
before CSR, novices accounted for about 14% 
of K-3 teachers in schools with the smallest 
proportion of low-income students and 22% 
in schools with the largest proportion of low- 
income students (see Figure 9). By i 997-98, 
nearly one in three teachers in schools with 
the highest percentage of low-income stu- 
dents were novices compared with one in 
four teachers in the lowest percentage of low- 
income students. But the difference in the per- 



Worsening of Teacher Inequities 

Cause for Concern 

Information from 1 995-96 about pre-CSR 
conditions shows what many educators knew 
or suspected for a long time: schools that 
were larger, urban, or with higher concentra- 
tions of low-income or ELL students had the 
most difficult time attracting credentialed 
teaching candidates. CSR appears to have 
made that situation worse. 

Exactly why the shift occurred is unknown; 
the first-year evaluation did not examine how 
teachers ended up where they did. Initial 
analysis shows that novice teachers who had 
the most education and full credentials chose 
what might be considered more desirable 
schools — ^that is, those with fewer low- 
income, ELL, Hispanic, or minority students. 
More qualified new teachers also appeared to 
prefer smaller, rural, or suburban schools. 
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Teaching practices 

DIFFERED LIHLE 

Since few studies about class size reduction 
have looked closely at classroom activities, the 
evaluators decided to examine underlying 
assumptions about the impact of CSR on teach- 
ers’ classroom practices. Are teachers doing 
something different? Or are they just doing 
more of what they were doing before CSR? 

Using a survey and case studies, the evalua- 
tors focused specifically on curriculum and 
classroom practices in mathematics and lan- 
guage arts.They explored how smaller classes 
might be different One such idea is that the 
curriculum is more rapidly covered so that 
more topics are taught Another is that the 
same content is covered, but more extensively. 
A third is that instructional practices — how stu- 
dents are grouped or dealt with individually — 
are altered In some Important way. 



Content Coverage Was 
Generally the Same 

Neither the survey nor the case studies 
revealed clear differences in content covered 
in smaller and larger classes. Analysis indi- 
cates that teachers in both settings taught 
similar numbers and types of topics. In mathe- 
matics, for instance, teachers in the larger 
classes covered 90 (of I 1 6) topics, and teach- 
ers in reduced size classes covered 99. 
Virtually the same number was covered in 
language arts too. Teachers In both classroom 
settings also overlapped considerably in the 
topics where they spent the most time (e.g., 
multiplication tables) and that they covered 
most frequently (e.g., reading and writing). 



The evaluation found a few differences in 
the extent to which topics were covered. In 
mathematics, for example, therejs evidence 
that teachers in larger classes spent in excess 
of ten hours per topic on twice as many of 
the curricular topics as did teachers in smaller 
classes. Although why is not known with cer- 
tainty, one explanation could be that having 
more students means that teachers must take 
more time to bring each one along to a desir- 
able level of performance. But it may also 
mean that these teachers had a higher stan- 
^or performance. 





HOW TEACHERS’ CLASSROOM 
PRACTICES WERE MEASURED 



In exploring differences in classroom practice, the Consortium relied on 
two sources of information. One was a questionnaire about curriculum 
and classroom practices answered by 3^ grade teachers as part of the " 
1 998 teacher survey. Evaluators found that the characteristics (e.g., creden- 
tial status) of teachers in reduced and regular sized classes and those of 
their students (e.g., language ability) were similar, so that reported differ- 
ences in teaching practices were not a result of these factors. Generally, 
the degree of professional development among both groups of teachers 
also was comparable, except that teachers in reduced size classrooms 
received more CSR-specific training, in accordance with state mandates. 

Second, the Consortium conducted case studies of 16 3^ grade teachers in 
the greater San Francisco and Los Angeles metropolitan areas. Half of them 
were teaching 20 students and half of them 30 or more. The selected 
teachers and the classrooms represented a diversity of backgrounds and 
student populations. Each case study included a survey of curriculum con- 
tent, face-to-face interviews, daily logs completed by each teacher for two 
weeks, and observations and videotapes. The classroom practices section of 
the evaluation was also funded by the U.S. Department of Education. 



CSR Teachers Say They Do More 
of Most Activities 

Teachers in smaller and larger classes general- 
ly spent about the same number of total min- 
utes on mathematics and language arts.There 
were small differences in the frequency of 
various student activities in these subjects 
(see Figures 10 and I I ), especially the more 
complex ones. 

Students in CSR classes spent more time 
writing narrative pieces, for example, or play- 
ing math games and using patterns to find 
mathematical relationships. Further, observa- 
tions in the case studies indicated that teach- 
ers in smaller classes were involved in more 
activities consistent with mathematics teaching 
reforms, such as writing about math and col- 
lecting or analyzing data. 



With minor 
exceptions, teach- 
ing practices were 
remarkably simi- 
lar in reduced 
and regular sized 
classrooms. 



CSR Teachers Spent Less Tome 

Instructing Whole Classes 

Both the survey and case study data reveal 
that class size had an impact on how students 
were grouped for instruction: group size cor- 
responded to class size. Generally, teachers in 
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FREQUENCY OF STUDENT 
ACTIYBTIES; MATHEMATICS 



Figure 1 1 



FREQUENCY OF STUDENT 
ACTIVITIES; UNGUAGE ARTS 
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Teaching practices in mathematics and language arts were similar in reduced and non-reduced size classes, although CSR teachers did 
somewhat more of most activities. 



Note: None of the differences between reduced and non-reduced classes is statistically significant. 
Source: RAND Daily Log, 1998 
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Figure 12 


GROUPING PRACTICES: 


Figure 13 


GROUPING PRACTICES: 




MATHEMATICS 


LANGUAGE ARTS 





INon-Reduced 
cm Reduced 




Group Practice 

In both mathematics and language arts, teachers in reduced size classes spent more time with smaller groups of student than teachers in 
non-reduced size classes. 



‘Note: Differences between reduced and non-reduced class are significant at the 0.05 level. 



"rce: CSR Consortium 1 998 Survey of Teachers 
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Figure 14' 



larger classes 
spent more time 
conducting lessons 
for the whole 
class or larger 
groups of five or 
more students. 

Teachers in small- 
er classes spent 
more time with 
smaller groups and 
giving students 
individual atten- 
tion (see Figures 
1 2 and 1 3). 

CSR Teachers 
Spent a Little 
More Time 
with Problem 
Readers 

The results are 
somewhat mixed 
when it comes to 
individualized 
instruction. One 

presumed benefit of reducing class size is the 
opportunity for teachers to work more 
directly and closely with individual students. 
Indeed, the case study teachers in smaller 
classes spent more time with students one- 
on-one.Yet the survey data, which are consid- 
ered more reliable because they represent 
the practices of 672 as opposed to 16 teach- 
ers, indicate that teachers in both types of 
classes spent on average about the same time 
working individually with students in language 
arts and mathematics. 



TEACHER TIME SPENT ON SELECTED ACTIVITIES 



^Disciplining students 











1 













Non-Reduced 
[ " 3 Reduced 



Monitoring/assessing progress 
Doing paperwork 
Diagnosing individual learning needs 

Reviewing homework 

Providing individual feedback 
(oral or written) on student work 

^Addressing individual 
students' personal concerns 

Discussing student-initiated topics that 
were ndt part of the planned lesson 



Average Hours per Week 

Teachers in reduced size classes spent more time diagnosing individual students' learning needs and addressing 
their personal concerns. Other teacher activities were similar for reduced and non-reduced size classes. 

*Note: Differences between reduced and non-reduced classes are statistically significant at the 0.05 level. 

Source: CSR Consortium 1998 Survey of Teachers 6/99 




CSR Teachers Spent Less Time 
Disciplining and More Time 
Addressing Student Needs 

Survey results also indicate two other ways 
that teachers in smaller classes spent more 
time with individual students: diagnosing indi- 
vidual students* learning needs and addressing 
their personal concerns. By contrast, teachers 
in larger classes spent more time disciplining 
students (see Figure 1 4). 



When asked specifically about time spent 
with poor readers, teachers in reduced size 
classes reported giving more individualized 
attention (five minutes or more). But that 
extra time was small: on average, poor readers 
received individualized help three times a week 
in smaller classes compared to two and a half 
times a week in larger classes. 



Teaching Practices Raise Questions 

for Future Research 

While these findings indicate few dramatic dif- 
ferences between newly reduced and regular 
sized classrooms, differences may emerge 
over time. For example, professional develop- 
ment for teachers in reduced size classes was 
not yet fully implemented. Perhaps teachers 
need more time to switch from covering cur- 
riculum in a way they were more accustomed 
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to when teaching larger classes. Future survey 
and case study data may reveal if and why 
such changes in teaching practices occur. So, 
too, could yet-to-be analyzed video observa- 
tions of teachers in their classrooms. 



Parents of 
students in 
reduced classes 
had more contact 
with teachers 
and were more 
satisfied with 
their children's 



In addition, no well-developed theory sug- 
gests why teaching in smaller classes should 
be different than in larger classes. Differences 
in student learning could be due to factors 
other than how content is covered or the fre- 
quency of learning activities. One possibility, 
for example, is that in smaller classes the 
social environment could be more conducive 
to supporting student learning, influencing it 
in sulstle ways. Such ideas will be explored in 
future analyses. 

Because of limitations with the data, this 
evaluation is unable to examine the relation- 
ship between the teaching practices of individ- 
ual teachers and the achievement results of 
their students. 



education. 

CSR PARENTS EXPRESS 
MORE SATISFACTION 
WITH CHILDREN’S 
EDUCATION 

Many educators believe parents who are 
more engaged in their children’s education 
will help their children succeed academically. 
Thus, an important evaluation question was 
whether or not CSR improves or encourages 
parental involvement. 



HOW PARENTS WERE SURVEYED 




Another evaluation question was what effect the class size reduction pro- 
grann had on parents’ involvement with their children at home and in the 
classroom, as well as their attitude toward this multi-billion dollar invest- 
ment The Consortium sent a questionnaire to a sample of two parents of 
3'^ graders, each of whose teachers also were participating In the survey. 

Of the 2, 1 12 parents who were contacted, 1,075 responded (or 50,9%), 
Slightly more than half had children in the smaller classes,The characteristics 
of the parents in the two groups were generally similar — with the excep- 
tion that the parents of the CSR 3 ^^ grade students were slightly more like- 
ly to be college graduates and to have higher incomes. 
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Slightly More Contact, Yet No 
Significant Change in Parental 
Involvement 

The evaluators surveyed parents of 3'^ 
grade students in larger and smaller classes. 
Responses indicate that smaller classes may 
modestly increase interactions between par- 
ents and teachers. While 74% of parents in 
CSR classrooms reported initiating contact 
with their child’s teacher, 69% of non-CSR 
parents did so. Similarly, teachers contacted a 
larger percentage of CSR parents at least 
once during the school year than parents of 
students in larger classes. 

However, the frequency with which par- 
ents volunteered in the classroom did not dif- 
fer significantly between the two parent 
groups. Nor were there differences in the 
extent to which parents were engaged with 
their children’s schooling at home. 



Parents Gave Smaller Classes and 
Their Teachers High Marks... 

The size of their children’s class was associat- 
ed with parents’ overall satisfaction with their 
local schools. Parents of 3"* graders in smaller 
classes gave higher ratings to every aspect of 
their children’s schooling. In addition, they 
were considerably more positive about the 
size of their child’s class than parents whose 
students were in regular size classrooms. All 
the parents rated access to special programs, 
counseling, and computers between good and 
very good, as Figure 1 5 shows. 

Both sets of parents gave the most posi- 
tive rating to the qualifications of their child’s 
teacher, closely followed by the teacher’s abili- 
ty to maintain discipline. As discussed previ- 
ously, the overall qualifications of California’s 
large teaching force have diminished post- 
CSR. Whether the teacher was new to the 
profession or experienced made virtually no 
difference, however, to parents in rating 
teacher qualifications. 

When asked how satisfied they were with 
the overall education of their children, par- 
ents’ opinions differed very little across 
demographic groups. African-American par- 
ents of students in both CSR and non-CSR 
classes gave the lowest ratings of any ethnic 
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group to this question. 
But consistent with all 
parents’ responses, 
those with children in 
CSR classes gave a 
higher satisfoction 
ranking than parents in 
non-CSR classes. 



...Yet They 
Recognized the 
Importance of 
Other Programs 
Too 

The parent survey also 
asked whether some of 
the money for smaller 
classes would be better 
spent on different edu- 
cation programs.The 
respondents supported 
reallocating some CSR 
funds, although parents 
with a child in a smaller 
class were less likely to 
do so (see Figure 1 6). 
The top priorities dove- 
tailed with the areas 
that received the lowest 
satisfaction ratings, such 
as equipping all schools 
with computers and 
providing extra support 
for students. The latter 
included tutors for indi- 
vidual children and 
enrichment programs 
after school. 

Other programs 
favored by parents 
included expanding 
music and arts pro- 
grams and providing 
scholarships to all stu- 
dents who qualify to go 
to college. As would 
be expected, parents in 
reduced size classes 
were least enthusiastic 
about the option of 
Q ding CSR funds 
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Figure 15 



PARENTS’ RAT8NGS OF THE5R CHILD’S EDUCATION 




Poor 



Fair 



Good Very Good Excellent 



Parents of children In reduced classes had more positive opinions of many aspects of the education program. 

Note: All differences between ratings from non-reduced and reduced size classes are significant at the 0.05 level with the excep- 
tion of Teacher’s Qualifications." 

Source: CSR Consortium 1998 Survey of Parents 6/99 



PARENTS’ RATINGS OF ALTERNATIVE 
USES OF SOME OR ALL CSR FUNDS 



Reduce class size from 
30 to 27 in a(l 12 grades 

Expand sports programs 

Provide, summer 
school for ali children 



Increase teachers' salary 

Upgrade the training 
of the teachers 

Expand music and 
arts programs 

Provide scholarships 
to all who qualify to go to college 

Provide enrichment 
after-school programs at all levels 

Tutor individual 
children having difficulties 

Equip all schools with computers 
and train teachers to use them 




Non-Reduced 



EH Reduced 



- 2-1 0 *1 *2 
Strongly Disagree Disagree Uncertain Agree Strongly Agree 



Parents of children in reduced size classes were less agreeable to spending CSR funds on alternative 
programs than parents of students in non-reduced size classes. 

Note: All differences between ratings from non-reduced and reduced size classes are significant at the 0.05 ievei. 



Source: CSR Consortium 1998 Survey of Parents 
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evenly throughout grades K- 1 2 in order to 
reduce average class sizes from 30 to 27. 
Parents whose children were still in larger 
classes indicated more support for this 
alternative. 



Not All Parents Know 
About CSR... 

Both parents and educators indicated that 
parents had little influence over local deci- 
sions about when and how CSR was imple- 
mented, a finding that could be attributed to 
the short lead time for starting the program. 

Three-quarters of parents with students in 
reduced or non-reduced size classes were 
aware of CSR. However, about one-fourth of 
the grade parents surveyed said they had 
not heard of CSR at all, including 21 % of those 
with children in the smaller classes. Only 
about 10% had attended a meeting about CSR. 
Those reporting no knowledge of the pro- 
gram were generally less well educated 



Figure 17 



100 



PERCENTAGE OF 3*^ GRADE STUDENTS 
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After adjustment for characteristics of the students, the gain in the percentage of 3^ 
graders above the national average associated with CSR was small. 



Source: CSR Consortium analysis of Student Assessment and Reporting (STAR) public release 
data for 1 997-98 6/99 
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and more likely to be foreign born and had 
lower incomes, a circumstance that probably 
holds in other aspects of parental familiarity 
with school operations. 



...But They Were 

Generally Satisfied 

Whether or not the positive response of a 
parent to CSR will stimulate a closer connec- 
tion with his or her child, and in turn lead to 
improved student performance, is unknown 
for now. But satisfaction may count for some- 
thing: three-quarters of the surveyed superin- 
tendents and principals reported that 
complaints about K-3 teachers had not risen 
since the smaller classes were formed, and in 
fact one-quarter said the number of com- 
plaints had dropped. 

Small achievement 

GAINS FOR ALL STUDENTS 

Early Results Show Small, 

P’ositive Gains 

This aspect of the Consortium’s evaluation is 
the most anticipated.The results so far are 
positive: 3''^ grade students in smaller classes 
did, on average, have higher test scores on a 
statewide test in reading, language arts, and 
mathematics than students in larger classes. 
These differences, however, were small. 

Because of the speed with which schools 
reduced I and 2"^ grade class sizes, 3'^ grade 
provided the only opportunity to compare ade- 
quate numbers of students who had been in 
smaller classes with their peers who had not. 
Differences that existed between the two sets 
of students prior to CSR were factored out. 

In every case but spelling, the students in 
CSR schools scored better than students in 
non-CSR schools, even when the scores were 
recalculated to eliminate pre-existing differ- 
ences between the two groups.The "effect 
size" of the difference between students in 
smaller and larger classes was nearly 0. 1 , or 
one-tenth of a standard deviation.That is 
equivalent to a 2 to 3 point gain on average in 
the scale score on the Stanford Achievement 
Test, Version 9 (SAT-9). Put another way, this 








gain would move a student who had scored at 
the 50^ national percentile rank to about the 
53'^ percentile rank. 

“Effect size” is a standardized measure of 
the difference in test scores between two 
groups.The larger the effect size, the larger 
the difference.The word “effect,” however, 
does not imply a causal relationship. 



As noted earlier, some possible reasons for 
the discrepancies between the two states are: 

■ Classes in Tennessee’s controlled experiment 
were 1 3 to 1 7 students per teacher, not 20. 

■ In California, the CSR schools with the high- 
est percentage of disadvantaged students 

also had the highest percentage of uncre- tl- j 
dentialed and new teachers. ' Sroders in 



Figure 1 7 shows the adjusted results by 
subject. Here the SAT-9 scores are converted 
to the way the California Department of 
Education displays them, i.e., in terms of the 
percentage of California students who score 
above the 50^ national percentile.These num- 
bers reveal the same pattern as above. 

To address concerns about the validity of 
the test for ELL students, evaluators looked at 
comparisons between reduced and non- 
reduced size classes without ELL students’ 
scores. Although the scores of both groups 
then went up, the relative gains associated 
with CSR stayed the same. 



smaller classes 

■ Nearly all students in Tennessee spoke ^ r u k 

English, while in California nearly one out of scored slightly 
three was an ELL student. better than 3^ 



How significant is CSR’s relationship to stu- 
dent achievement compared with that of other 
variables? To gain that perspective, evaluators 
looked at comparisons between white and 
African-American, Hispanic and Asian students; 
ELL students and fluent English speakers; stu- 
dents from lowest and highest income families; 
and students in reduced and non-reduced size 
classes.These comparisons (see Figure 18) 
show that the differences associated with CSR 
are about one-eighth to one-tenth those asso- 
ciated with these other variables. 



graders in larger 
ones in reading, 
language, and 
math. 



Students in Smaller 
Classes All Gained 
the Same 

Overall, the gains were similar 
for all 3'^ grade students in 
the CSR schools, regardless of 
background. Given California’s 
large number of low-income, 
minority, and ELL students, 
the across-the-board im- 
provement is good news. 

However, these findings 
are inconsistent with the 
Tennessee experiment that so 
influenced California legisla- 
tors.There,the achievement 
gains were more than twice 
the size of these early results 
in California. And in Ten- 
nessee, the more disadvan- 
taged students experienced 
the greatest growth, a differ- 
ence that thus far has not 
been observed in California. 




“EFFECT SIZES” ASSOCIATED WITH THIRD GRADE 
STUDENTS’ BACKGROUND CHARACTERISTICS 
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The effect size associated with CSR was significantly smaller than the effect size associated with 
other student background characteristics. 

‘Note: Average from 4^ and 5*^ grade reference groups. 

Source: CSR Consortium analysis of Student Assessment and Reporting (STAR) public release data for 1997-98 6/99 
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ANALYSIS HAMPERED BY WEAK 

OR NON-EXISTEE^T DATA 




Because of the high expectation that smaller classes would lead to higher test scores, 
particularly for less advantaged students, measuring achievement was a paramount 
task of the evaluation. 

To determine CSR's relationship to achievement, the Consortium faced several 
methodological challenges: 

■ First, California had no baseline statewide test scores against which to compare stu- 
dent achievement under CSR. Between 1 994 and 1 998, students took a variety of 
tests that were chosen by their local school district. A statewide exam, the Stanford 
Achievement Test, Version 9 (SAT-9), was finally given in spring 1 998, the second 
year of the class size reduction program and the first year of the evaluation. Since 
the test was new, achievement growth from one year to the next could not be 
tracked.The 1 999 test results are expected to be available in late June. 

■ Second, unlike Tennessee, California offered funding for smaller classes to every 
school district in the state. Most took the offer and formed small classes in I ^ and 
2 nd grades almost immediately Comparison of CSR and non-CSR test scores could, 
therefore, only be made for 3*^ graders, since kindergarten students are not tested. 

■ Finally, the 3*^ graders in CSR classes had different characteristics from those in 
non-CSR classes.The districts that had not yet formed smaller classes were more 
urban, with higher percentages of low-income, minority, and ELL students. The eval- 
uators knew that achievement differences could have as much or more to do with 
the differences among students as with the size of their classes. Therefore, any fair 
comparison of test scores between CSR and non-CSR students had to adjust for 
the differences in student achievement that were related to other student back- 
ground variables. Evaluators assumed that the socioeconomic makeup of 3'^ 
graders in a school would be similar to that of the 4^ and 5^ graders in the same 
school and would have a similar impact on test scores. They adjusted their estimate 
of the CSR effect based on the differences in scores from these older students who 
were not exposed to CSR 



The data for this part of the evaluation came from CBEDS and the SAT-9 administered 
in spring 1998 as part of California's Standardized Testing and Reporting (STAR) pro- 
gram. In addition to the challenges listed above, some data were missing and some 
were clearly incomplete or incorrect. 



Given that CSR was implemented 
so rapidly, most schools will have 
fully phased in CSR by the end of the 
third year. That will make it impossi- 
ble to find comparison groups of stu- 
dents without CSR as was done this 
year. In the future, the evaluators 
propose to study CSR’s effect on 
achievement by examining trends in 
achievement in successive cohorts of 
4^ and 5* grade students. If CSR is 
working, achievement scores in these 
groups should increase from year to 
year. Special attention will be paid to 
differences in schools with higher 
percentages of students with minori- 
ty or low-income status or who are 
learning English. 

In short, initial achievement 
results are positive but small. A more 
complete picture of CSR’s relation- 
ship to student learning will not be 
available for several more years. 

Policy 

IMPLICATIONS AND 
NEXT STEPS 

CSR: Cause for Cautious 
Optimism or 
Disappointment? 
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Examining CSR’s Relationship to 
Achievement Is Complex 

CSR is only one of many reforms that 
California schools are trying to implement 
simultaneously, some of them mandated and 
others adopted voluntarily. Other circum- 
stances could account for some of the "CSR 
effect," so the student achievement gains in 
smaller classes should probably be attributed 
to CSR and an unknown combination of 
other reforms. 
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These initial achievement findings 
could be viewed as cause for cau- 
tious optimism. Standardized test 
scores show a small positive gain for 
students in smaller classes over 
those not In the smaller classes. Yet 
unknown is whether the positive difference in 
student achievement will be sustained or will 
grow and whether it is even the CSR program 
that was a cause of this gain. 

Given the size of the Investment in CSR — 
$ I to $ 1 .5 billion in the first two years — 
some may see these findings as disappointing. 
Moreover, the achievement gap between tra- 
ditionally at-risk students and their peers 
remains unchanged. Low-income, minority, 
and ELL students did not do significantly bet- 
ter than other groups of students. 
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Further, these early findings confirm 
anecdotal reports about the unintended 
consequences of CSR’s rapid and widespread 
implementation.While the majority of Califor- 
nia’s unified and elementary school districts 
struggled with aspects of creating smaller 
classes, some districts suffered more than 
others.Teacher hiring and resource inequities 
that existed prior to CSR became worse after 
it was implemented. 

Cause and Effect Claims 
Are Premature... 

One challenge facing the Consortium, and 
others, is disentangling CSR’s unique contribu- 
tion relative to other reforms. Many existing 
and newly-created reforms are also part of 
California’s education system. What part of 
any improvement — or decline — in achieve- 
ment results can be attributed solely to CSR, 
therefore, remains open to further study. 
Direct inferences about the relationship of 
the achievement gains to other factors, such 
as changes in teacher qualifications, changes in 
classroom practices, or interaction with other 
reforms, are intriguing but premature to make 
at this juncture. 

...But Mid-Course Adjustments 
May Be in Order 

Even with these ambiguities, the CSR 
Research Consortium’s initial findings do raise 
important questions about the effectiveness 
of the effort and the way in which it has been 
implemented. Education policy makers have 
the opportunity to review these formative 
implications which suggest mid-course adjust^ 
ments that would improve the effectiveness of 
the continued implementation of CSR in the 
early grades, as well as any future expansion 
of the program to other grades. 

Bedydng Space and 

Facilities Problems 
■ More Facilities Resources May Be 
Needed to Ensure All Students Receive 
Full Benefit from CSR. The shortage of 
facilities was the single most important 
deterrent to implementing CSR fully and 
Q"^kly. Despite the additional facilities 



funding during CSR’s first two years, some 
districts were still unable to take full advan- 
tage of the program, denying some stu- 
dents — mostly low-income and minority — 
access to smaller classes. Without more 
money for ^cilities, space may still be taken 
from other programs. 

Addressing the Decline in 
Teacher Preparation 

■ Findings Underscore State Priority on 
Improving the Teacher Preparation and 
Development System. The increase in K-3 
teachers who are not fully credentialed is 
cause for concern. Fortunately, state policy 
efforts are already under way to rectify this 
situation, across all grades.The Consor- 
tium’s findings lend further urgency to 
recent state efforts to increase the supply 
of qualified teachers and improve their 
preparation. 

■ Teacher Incentive and Placement 
Policies Need Reexamination. Employ- 
ment policies at the local level, such as 
financial incentives and collective bargaining 
rules that affect the placement of teachers, 
may need to be reexamined along with 
strategies to attract qualified teachers to 
schools most in need of their expertise. 
Which strategies to pursue should be 
informed by further study of the needs of 
teachers and schools and the capacity of 
the existing system to meet them. 

■ Special Attention Needed on Effective 
Training for Uncredentialed Teachers in 
CSR Classrooms. Similarly, more informa- 
tion is needed about the level, intensity, and 
quality of on-the-job support and training 
for those teachers in CSR classrooms who 
lack full credentials.This information could 
ensure that state and district resources are 
targeted effectively for these teachers, 
many more of whom are now working in 
all schools across the state. 

SmpmvQinig Teac&HDinig omi 

SmaiSer Classes 

■ More Clarification Needed on Effective 
CSR Classroom Practices. The Consor- 
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tium’s research to date shows slight differ- 
ences in the way curriculum is covered and 
the types of activities teachers ask students 
to engage in.These teaching practices 
require further exploration so that designers 
of professional development programs know 
what new skills teachers should have and 
how best to teach content in CSR class- 
rooms to maximize the effect on student 
learning. At a minimum, the Consortium 
would like to know for certain whether 
CSR’s impact has to do with classroom prac- 
tices or with something yet undiscovered. 

Increasing Program Flexibility 

■ Districts Need Greater Spending 
Flexibility. The cost of CSR varied accord- 
ing to differing local constraints and unique 
problems faced by each district. Greater 
flexibility in using CSR funds could allow 
districts to direct money where it is need- 
ed the most or to address related imple- 
mentation barriers, such as building more 
facilities or hiring better qualified teachers. 

■ Funding Formula May Need to Be 
Reconsidered. Each district had different 
costs associated with implementing CSR 
fully. State resources could have been more 
effective In ameliorating those cost differen- 
tials if they had taken into account individ- 
ual district needs. 

Examining Cost-Effectiveness 

■ Analysis of Overall Costs Needed. 

Likewise, it is too early to estimate the full 
cost of the reform. Despite the allocation of 
about $ I to $ 1 .5 billion per year, the costs to 
some districts have been greater and were 
not borne equally. It will take time to deter- 
mine what those costs have been to certain 
communities, but for now It is clear that the 
differences deserve further attention. 

■ Summary Judgments about Achievement 

Gains Premature. It is too soon to know 
whether the “small positive effect” of CSR 
is evidence of success. Given the newness 
of the program, achievement results In the 
next year or two will provide better insight 



into whether CSR is cause for celebration 
or reconsideration. 

Planning Future CSR Policy 

B Thoughtful Planning Needed to Guide 
Future CSR Policy Decisions. In addition 
to weighing costs and benefits, policy makers 
should have a list of “decision rules” that 
allows them to consider all the advantages 
and disadvantages to expanding or modifying 
existing CSR policy. Among the issues to be 
considered are the pace of implementation, 
teacher supply and demand, and targeting of 
resources to students most in need. Such a 
set of decision rules could be used by other 
states that are considering CSR too. 

■ CSR Must Be Part of Coherent Set of 
Policies That Ensures All Students Benefit 
California is in the midst of trying to bring 
more coherence to its educational policy 
making. Efforts to align curriculum and 
assessment to standards are an Important 
part of this process. Likewise, the Legislative 
Analyst’s Office (LAO) called for the devel- 
opment of a master plan that would review 
state educational policies and suggest roles 
for different levels of government. 

In a similar vein, policy makers should care- 
fully consider CSR’s role In any nevy com- 
prehensive strategy for education and 
should think about how new policies will 
interact with CSR. An important issue is 
whether policies, singly or in combination, 
benefit all students and schools equally. 
Who gains and who loses are politically 
unpopular but critical questions to ask. 
Many districts have put so many resources 
into CSR (financial, human, and physical 
resources) that they may find it difficult to 
implement other educational reforms. 
Urban districts, in particular, have been 
under considerable stress due to CSR. 
Given the multiplicity of program initiatives, 
it is important to monitor districts’ 
response to new p.»"ograms and watch for 
negative interactions. 

9 Cumulative Impact of Expanded CSR 
Needs Further Tracking. Although the 
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positive effects of CSR on achievement may 
be viewed as small, 6,000 more 3*^ grade 
' students are above the national median 
score in reading and about 9,000 more are 
above the national median in math. More 
information about the cumulative impact of 
CSR on achievement will be tracked during 
the next three years. 

■ Other Potential Benefits of CSR Should 
Also Be Analyzed. Beyond achievement, 
other advantages or positive outcomes 
related to CSR should be carefully tracked 
too, such as increased parent and teacher 
satisfaction. 



Enhancing the State’s Ability to 
Evaluate CSR and Other Reforms 

■ A Statewide Student Information 
System Is Long Overdue. The CSR evalua- 
tion is limited by the lack of systematically 
collected, high-quality statewide informa- 
tion. Improvements to the state education 
data system would assist in providing better 
information about the impact of different 
educational reforms. Such a system should 
follow individual students over time, link 
the performance of students to the qualifi- 
cations of their teacher, and track informa- 
tion about implementation of the panoply 
of state reforms. 

■ Evaluations Should Begin Before 
Reforms Are Implemented. The inability to 
collect baseline data for assessing the 
impact of a reform hampers any evaluation 
effort While every policy does not need to 
be evaluated from the onset reforms of this 
magnitude are worthy of study and good 
evaluations require enough advance start-up 
time so that baseline data can be collected. 



Conclusion: CSR Still a 

Work in Progress 

Since CSR is a work in progress, the Consor- 
tium’s evaluation will continue for three more 
years. Clear successes include almost universal- 
ly smaller classes for I “ and 2"'’ graders and 
widespread reductions for kindergarten and 
3"* grade.The small improvements in student 
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achievement have occurred despite the prob- 
lems of finding space and qualified teachers. 
These shortages were felt the most in schools 
with higher proportions of low-income, minor- 
ity, or ELL students. 



Policy makers need not wait three more 
years to attend to these problems.They can 
also consider how CSR interacts with other 
reform initiatives and the need for better 
data. Eventually, the Consortium hopes to 
provide them with a better understanding of 
the effects of CSR on all students and 
whether or not the benefits of the program 
are worth the cost, g 
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